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ABSTRACT 


Many  areas  of  stochastic  control  were  studied  in  this  period. 
Monte-Carlo  algorithms  of  type  appearing  in  adaptive  systems  were 
studied  and  stability,  convergence  and  rate  of  convergence  results 
obtained.  The  analytical  techniques  developed  for  this  purpose 
have  a  wide  applicability  in  the  study  of  similar  algorithms. 

Robust  computationally  oriented  ,'prox imat ions  to  optimal  non-linear 
filters  were  developed.  A  deep  study  of  how  to  find  the  Markov 
system  that  best  approximates  a  given  non-linear  system  with  a 
wide  band  -width  input  was  undertaken  and  very  good  results  ob¬ 
tained.  The  problem  is  important  in  control  and  communication 
theory  because  it  is  usually  desired  to  use  the  many  available 
methods  for  analvring  Markov  systems.  The  results  obtained  seem 
to  be  the  best  available  so  far  and  applications  to  control  and 
communication  theory  arc  being  pursued.  Optimal  stochastic  control 
problems  under  the  condition  of  partial  information  on  the  system 
state  were  studied  and  a  new  formulation,  similar  to  that  of  the 
non-linear  filtering  problem,  developed.  The  approach  seems 
promising  in  providing  qualitative  information  on  this  difficult 
class  of  problems.  Part  of  the  technique  uses  measure -valued  processes 
in  a  way  which  has  wider  applicability  to  problems  involving  dis¬ 
tributed  states  or  information. 
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SUMMARY 

H.  J.  Kushner  continued  his  investigations  into  numerical 

and  recursive  monte-carlo  methods.  One  particularly  important 

is 

result/ (2],  where  both  convergence  and  rate  of  convergence  and 
stability  results  are  developed  for  stochastic  approximations 
of  the  type  Xn^j  •  Xn  ♦  ah(Xn»£n),  for  small  a,  where  {£n> 
is  a  rather  arbitrarily  correlated  noise  sequence.  This  type  of 
algorithm  occurs,  or  is  heavily  used,  in  many  adaptive  control 
and  communication  systems,  adaptive  filters  and  identification 
methods.  The  results  are  the  first  of  their  type  and  shed  con¬ 
siderable  light  on  these  procedures  under  realistic  conditions. 

In  this  work  the  techniques  of  (1]  were  extended,  to  cover  the 
practically  important  cases  dealt  with  in  (ZJ.  Many  new  ideas 
and  applications  arc  currently  being  developed. 

The  problem  of  robustness  of  non-linear  filters  is  an  old 
and  important  one,  but  it  is  only  recently  that  significant  results 
have  become  available.  Reference  (S)  by  Kushner  developes  an 
approximately  optimal  computational  method  for  non-linear  filtering. 
This  filter  has  particularly  nice  properties  in  that  the  estimate 
is  a  continuous  function  in  the  observational  data,  uniformly  in 
the  approximation  parameter.  This  smoothness  is  particularly  im¬ 
portant  in  applications,  since  it  is  important  that  the  filter  not 
be  oversensitive  to  modelling  assumptions. 

A  particularly  interesting  and  practically  important  area  of 
research  was  initiated  by  Kushner  (4),  [Sj.  These  references 
deal  with  the  problem  of  finding  the  closest  'nice'  Markov  system 
to  a  particular  given  nonlinear  stochastic  system,  with  wide 
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band  -width  noise  inputs.  These  problems  occur  frequently  in 
both  control  and  communication  theory,  and  it  is  quite  common, 
often  using  purely  formal  methods,  to  try  to  find  a  nice  Markov 
approximation,  so  that  the  powerful  associated  analytical  methods 
can  be  used.  References  |4),  (5)  give  the  best  currently  avail¬ 
able  methods  and  conditions.  The  techniques  are  also  simpler 
than  those  used  in  past  works. 

Some  of  the  numerous  applications  to  control  and  communica¬ 
tion  theory  arc  being  investigated  and,  so  far,  the  results  are 
very  nice.  This  latter  work  is  being  done  with  Professor  Y. 
Bar-Ness  of  Tel  Aviv  University  who  is  visiting  us  this  year. 
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W.  H.  Fleming;  continued  his  program  of  research  in  optimal 
stochastic  control,  with  complete  or  partial  observations,  and 
on  Markov  diffusion  models  of  distributed  parameter  stochastic 
systems.  A  new  direction  of  work  is  the  study  of  optimal  control 
of  partially  observed  stochastic  systems  using  techniques  of 
measure-valued  processes.  This  is  done  by  introducing  a  new 
"separated"  stochastic  control  problem  with  completely  observed 
states,  the  states  in  the  new  problem  being  probability  measures 
on  the  original  state  space.  The  original  and  separated  control 
problems  are  related  by  the  nonlinear  filter  equation  for  the 
conditional  state  distribution  given  the  observed  data.  By 
understanding  well  the  relation  between  these  two  control  problems, 
it  is  hoped  to  shed  light  on  some  basic  theoretical  questions  of 
long  standing.  One  of  these  questions  is  to  give  appropriate 
conditions  for  the  existence  of  optimal  controls  in  case  of 
partial  observations. 
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Mart  in  Day  visited  at  Brown  during  February  -  .June  1979. 

Me  recently  found  a  significant  improvement  of  Fleming's  stochastic 
control  method  (J.  Applied  Math.  Optimization  1978,  pp. 329-346) 
for  estimating  exit  probabilities  for  diffusions  depending  on 
small  parameters.  A  report  of  this  work  is  in  preparation.  An 
extension  of  t>  method  to  study  the  asymptotics  as  e  •*  0  of 
solutions  to  the  partial  differential  equation  cAu  ♦  b(x). 

Vu  ■  0  with  Dirichlct  boundary  data  is  currently  being  tried  by 
Day  and  Fleming. 
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